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ABSTRACT 


The United States has a tradition of military analysis using a federated or combined suite 
of models. However, these are not the only methods of modeling military problems. We 
consider the application and implications of foreign modeling approaches. The particular 
alternate technique we focus on is fuzzy comprehensive evaluation (FCE). FCE makes 
use of fuzzy mathematics, alone and in partnership with Analytic Hierarchy Process 
(AHP) models, to inform strategic and operational decisions. It is designed to aid leaders 
in capturing the complicated and sometimes “fuzzy” nature of multi-criteria decision 
problems through human knowledge and evaluations. These subjective inputs present 
criticisms regarding FCE solutions. FCE results are only as valid as the consistency of 
the subject matter expert’s opinions. Therefore, this thesis analyzes the FCE approach 
through a case study and evaluates the implications of FCE results when there is high 


variance in expert opinions. 


THIS PAGE INTENTIONALLY LEFT BLANK 


vi 


I. 


Il. 


IV. 


TABLE OF CONTENTS 


EN PRODUC DION ws sccssetessccsessstecsssiuccsshtetassassbsvessscehecspstedencisessessuavnssesouessdenestutensthuocess 1 

A. BACKGROUND wrssceuisedivcos test stucctdeccacseai sliced sieve teas vesapases Gancalbeciehinarsseceseianeies 1 

B. METHODOLOGY issccasccncesieactveds susscatvaneesdessackaanvancescesnuenqavoousesducsaneqansensguaesoes 2 

C. OCOBUEEC TV Ess ssseconsecsnasenactuctadi otessencscbssuccanncedenstuavaiesS0bvan actnetashsecdensaceooiedccinioees 3 

THE FUZZY COMPREHENSIVE ANALYSIS (FCE) METHOD..............scs000 5 

A. PUZZY COGIC virvccsctsssuessseiass nszctistiasues cansansetesscucenteedesenadyascevnnentisvauousnedcaneteeds 5 

B. ANALYTICAL HIERARCHY PROCESS ........ cc cescssscsscssecssscssescesssesenes 6 

CS soak tec ce eduens Sate sash conc otenanisatedn sansescuavnnsec Wise stucechate dasbuuctboebansddetnsaacentbesss 11 
1. Determine the Criteria Set and Develop the Evaluation 

GRAAL CHY 5 cassictciscnacssccesnis sosupescsscupavonsessec sovespncsuesiouccssoupacteseabecuasevensers 12 

2. AHP and Determining Element Weights.................scssssssssssessseees 12 

3. Establish Linguistic Grade Set and Evaluation Set...............sse000 12 

4. Evaluate the Alternatives’ Fuzzy Judgment Matrix...............ccs0e 12 

5. Calculate Overall Synthetic Result..............ccccsssccscsscsccsscsccessscesees 13 

BCE TN MILITARY LOGTS TICS \¢ ssiscssssssssdssossnaconsossncsssetnscssvessecssounvaderonsencsteesoavess 15 

A OPTIMIZATION YS F Chis cosseussessessverssasniaveanyoncenssnveondesussounsanstuspeenencssssse 15 

B DC IN ARR  gasel cos teak hist scccak tates Sek cs ieicese ashen iced Deseace a eteaevensden eis 16 

C MODELING ASSUMPTIONS ............cccsscssscsssccccssscscsssssecsescsssessessscssssseesees 17 

D OPTIMIZATION: MODEL a sscicssessgsnsenssasceessestossedentevsonagsessopssseesusachssnapbetones 21 

1. NCES stats tiga ts ssdccaSekuaveebtovedslen ata tecstiuuscleeassteneS aa pldsts Seatebeuenciatentinecss 21 

2. Data [UMits | Siscceslecsssstees scsi sd cdswivedecnstssicesesbedeostescleshecvasvensteshcssesbesesszescee 21 

35 DGCISION VAPiADlOS® sscisccss ss cvsctntscsssqacdstdxssosncecas sasessentausseopssadbesbadsenvenced 21 

4. BOP rrr Qh os secs cecasscesicdoctsnaessaasenssedacsuscceniencducecsapacassouxesdessascasareesiecads 21 

5. WDUSC US SHOU sta chek cots acstvnadd ote desusnetcvavtenecsudesateccd sate desasectuteusovscusenssencnioente 22 

6. TVET P'S OU UU ois Sepsciccss caccsucenthactshsseecstien senses iatebeathestaleetensceieraseovaineds’ 23 

E CT MO DE gag scesvestiusecenacdcapntsecsxecihsuctsoteacatavseccanbeCanscncstbauctontesseveutecdectebnacccs 24 

1. Determine Criteria Set and Develop Evaluation Hierarchy.......... 24 

2. AHP and Determine Element Weights .................sscsssssssssessssseseees 25 

3. Establish Linguistic Grade Set and Evaluation Set...............ss0000« 28 

4. Evaluate the Alternatives’ Fuzzy Judgment Matrix..............ccss0e 28 

5. Calculate Overall Synthetic Result..............ccccsscsccsscsscscsccssesceeees 30 

F. INSIGHTS AND CONCLUSIONS ...00........ccsscssscsccsscssescscscsseescesssesssssnesees 31 

HIGH VARIANCE IN EXPERT INPUT STUDY ......... ee ecsscsesceccencseneesesssesenes 35 

A. PRIZE UINC TION cso ures tssiucsasestnnessupvesusceasontesuavietasnsshvassennaaveneectacGessveceascevenes 36 

1. Set/TniGialize Parameters sicscsicecsssccscescesistessesedssesbeceesecteondessessecessseeees 36 

2: Import “True” Fuzzy Subjective Assessment Matrices ............... 37 

3. Function Implementation ...............cccscccccssccssscccssscccsssssccssecssssssseeees 37 

4. FR GSUIS scsciccascsaccseccsctcsiuh cenidecessinccshucecs Suess heat etisedsaveseseniveonaueseveneatabecced 38 

CONCLUSIONS dsp ccvscosencenseatctsdencecetegintethencccnesesteanse dvasuesbadGebnsisnssseucensesenosivecuvesadintesee 41 

A. BO UIVEIVEARR Y eSicciicte de scctes seevatens sevebe les csacte li seccatbuesaiest ite lebssndda elves dlascievaneossinases 41 


LIST OF REFERENCES 


INETEAG, DIS ERIBUTION TIS sncissisctessanisneaintastetsoteesiiestveceusaesssadansaeteleneessasenesncenebneekesanes 


vill 


Figure 1. 


Figure 2. 
Figure 3. 
Figure 4. 
Figure 5. 


LIST OF FIGURES 


Sample AHP hierarchy tree for alternative missile system selection 


POTD TEI caries se ies cos arith caictecptes i ueavs telspatd na daed oe aphek oa nalttva atacand nausis lected nvoaaies 8 
Transit scheme between a transfer point and a single base. ...........ceeeeeeeeeee 18 
Disposition of possible REB locations to a transfer point. ........... ce eeeeeeeeeees 19 
Losisti¢ resupply Wetarchy secs: oi idsislaceldandiazioatisclsssralaasstecgik ators aS 
RAB PSCUdO- COGS: veces secscs sk cpicgh stvvsavssoaseceucs decane suagaas aasqunmsoeneauuskseenea testoiosseaas 36 


ix 


THIS PAGE INTENTIONALLY LEFT BLANK 


Table 1. 
Table 2. 
Table 3. 
Table 4. 
Table 5. 
Table 6. 
Table 7. 
Table 8. 
Table 9. 
Table 10. 
Table 11. 
Table 12. 


LIST OF TABLES 


Sample pair-wise comparisons of Criteria 3’s sub-criteriOn...........ceeeeeeseeeeees 9 
Verbal description of 1 to 9 scale (from Zuiliang. et al., 1993)... ee 10 
Alternative logistical MOT parameters. ii. :2:.3:ccccasssedseaadersecns yeas nrasancwersseesies 20 
Parameter values for optimization Model «0.00.0... eeeeseeseeseeeeeeeeceseeneeeaeeeneeeeees 2D 
Pairwise comparisons of criteria with fractions converted to decimals.......... 26 
W CIS VOCLOR as s535s feria secceies ipacdhastiaweiaissccaxsbenages iy citeawlencigaaecan dl ite sance eaten 21 
(Notional) Subjective assessment Matrices ...........ccceecceeseeseeeteceteeeeeeeeeeenaeens 29 
FUZZY Ud Sent MALE CES ice cs vee ected releases leues coda ati tas Vow lovetatint tale os tes 30 
Overall synthetic evaluation results .........c.cccceccceceseceeeceeeceeseeceseceeeeeeeenseeesaeees 31 
Fuzzy evaluation with alternatives within the same category ............:cceeee 32 
CLS-5000 saved fuzzy evaluation results by number of experts.............0.0 38 
Scaled simulated fuzzy evaluation scores from 2 to 10 experts 0.00... cee 39 


x1 


THIS PAGE INTENTIONALLY LEFT BLANK 


Xii 


LIST OF ACRONYMS AND ABBREVIATIONS 


AHP analytic hierarchy process 

AO area of operation 

AOR area of responsibility 

CI consistency index 

CLF combat logistic force 

CLS-5000 Combat Logistic Ship (5000 tons) 
CR consistency ratio 

COA course of action 

FCE fuzzy comprehensive evaluation 
JCA joint campaign analysis 

JHSV Joint High-Speed Vessel 

LCS Littoral Combat Ship 

MCDM multiple criteria decision-making 
MLP Mobile Landing Platform 

MILP mixed integer linear program 
MOT means of transportation 

REB repositionable expeditionary base 
R2B robustness to bias 

SAG surface action group 

SCS South China Sea or S. China Sea 
ST short tons 


T-AKE USNS Lewis and Clark class dry cargo/ammunition ship 


xiii 


THIS PAGE INTENTIONALLY LEFT BLANK 


X1V 


EXECUTIVE SUMMARY 


The United States has a tradition of military analysis using a federated or combined suite 
of models. These are not the only methods of modeling military problems. We consider 
the application and implications of foreign modeling approaches. The particular alternate 
technique we focus on is fuzzy comprehensive evaluation (FCE). FCE makes use of 
fuzzy mathematics, alone and in partnership with the analytic hierarchy process (AHP), 
to inform strategic and operational decisions. In 1965, Lotfi A. Zadeh introduced fuzzy 
mathematics as a way to capture uncertainty, particularly when assessing decisions with 
qualitative factors. Fuzzy mathematics has not gained widespread use in the United 
States because other methods for dealing with uncertainty are preferred. Additionally, 
the AHP’s purpose is to organize and formulate weights for multiple criteria of a decision 
process. This thesis accomplishes three research goals: 
1. It critiques the application of fuzzy math and AHP in FCE. 


z. It finds differences between the FCE approach and current U.S. 
approaches when applied to military decision support processes. 


3. It evaluates the implications of FCE results when there exists a high 
variance in expert opinions. 


We examine the reasons why the results of optimization and FCE diverge. This 
proves useful for discovering weaknesses in the FCE results, as well as understanding 


places where there may be strategic miscalculations. 


We apply FCE to a military logistics resupply scenario and compare the results 
with a linear optimization model. The scenario presents three alternative means of 
transportation (MOT): the Independence variant Littoral Combat Ship (LCS-2), Joint 
High-Speed Vessel (JHSV), and a modified JHSV, Combat Logistic Ship (5000 tons) 
(CLS-5000), to supply the wartime demands of combat ships in theatre. The goal is to 
meet demands at the lowest operating cost of the MOTs. Although combinations of 
MOTs are possible solutions for the optimization program, the mixed integer linear 
program (MILP) selected nine CLS-5000s. The MILP was also able to rank the three 
alternatives from the lowest cost option to highest as follows: 1) CLS-5000 2) JHSV 3) 
LCS-2. FCE did not contain the ability to inform users of “how many” alternative ships 


XV 


were required to fulfill demand for this case study. Chapter III details these findings. 
Furthermore, we develop a simulated FCE study that expects to match the optimization 
results, and study how potential variation in expert opinion could lead to different or 


incorrect rankings. 


In Chapter III’s case study, we found a few criticisms for the FCE modeling 
technique. The main criticism is in its employment of human knowledge and evaluations 
for inputs. This method poses risks for introduction of bias and inaccurate results if not 
handled properly. Generally, the process to ensure the accuracy of expert judgments is 
that all judgments must be consistent. However, if expert judgments are truly incorrect 
but pass the consistency check, there is no real way of uncovering this discrepancy. The 
imprecision of results is also troublesome. FCE can handle the modeling of a few SMEs 
(subject matter experts). Although a realistic condition, this practice presents issues in 
precision due to the small sample size of expert input. Additionally, FCE only considers 
the mode of results as the solution and disregards the distribution of scores. For example, 
a fuzzy assessment with 52% of an alternative’s membership in the “fair” category and its 
other 48% in the “poor,” reflects only an alternative performing in the “fair” category. 
The almost equal membership of an alternative in the “poor” category is ignored. This 
reproachful handling of results is combined with contradicting techniques to handle 
alternatives within the same category. Unfortunately there is no guidance for FCE 
outcomes when one method of breaking ties presents conflicting results compared to 


another. 


Amidst these pitfalls, FCE presents an alternative decision-making approach that 
can handle smaller samples of data. This characteristic can prove beneficial when limited 
data requires analysis. Ballistic missile performance statistics are a prime example. 
Generally, sample experimentation of this kind is limited due to cost, resource, and 
policies of the manufacturer and consumer. Instead of waiting for limited or unattainable 
data, FCE acquires information from an alternative readily available source — experts. 
Furthermore, with enough experts, FCE’s approach to multi-criteria decision problems 
with subjective judgments can provide quantifiably useful and precise results as long as 
its pitfalls are considered. 


XV1 


Evidently, FCE results are as good as its subject matter expert inputs. Thus, we 
created and employed the R2B (robustness to bias) tester in Chapter IV to provide a 
generally applied method for testing the robustness of expert opinions. R2B tests high 
variance in subjective inputs (due to the bias in expert opinions) and evaluates the 
robustness in the subsequent FCE results. This thesis studied simulation replications for 
sequences of experts with the presence of one adversarial expert who provided intentional 
bias in each sequence. All simulation iterations involved complete execution of the FCE 
process. The selection scores for sequences of experts for each alternative were recorded. 
After the scores were tallied, they were compared to known “true” rankings of the 
alternatives. In the end, R2B confirmed that the same alternative would still be chosen 
even with added simulated variability and adversarial inputs. It also highlighted the ratio 
of good experts required for cogent outcomes when one adversarial expert was present. 
Moreover, even though results are specific and applicable to the scenario presented to 
R2B, this method can be applied to any set of expert opinions to estimate the potential 


effect of bias on FCE results. 
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I. INTRODUCTION 


I 


“Si vis pacem, para bellum—if you want peace, prepare for war.’ 


—Publius Flavius Vegetius Renatus 


A. BACKGROUND 


The task of preparing for armed conflict is extremely difficult and complex. This 
is evident in the face of belligerents who possess various means and ways of waging war. 
The number of contingency plans countering each possible scenario against all likely 
adversaries would be endless, the question of how to prepare for such events remains. 
Perhaps Sun Tzu’s teachings can provide enlightenment to this predicament. Sun Tzu 
(1963) imparted a famous proverb to his generals, “Know the enemy, know yourself; in a 
hundred battles you will never be in peril” (p. 84). His instructions to the warfighter were 
meant to impart the role of intelligence and assessment in framing strategies towards 
winning conflicts. The knowledge gained from understanding an opponent’s capabilities 
can lead to capitulation of the enemy through attacking these strategic sources of strength 
(i.e., command centers, forces, economy, leadership, etc.). Thus, possessing superior 


intelligence leads to advanced preparation. 


Even though two opponents may view the same problem differently, the one who 
most understands their opponent’s view gains the upper hand. Thus, this understanding 
poses risks when individuals incorrectly assess their opponent’s intentions and 
opportunities when an enemy’s capabilities are correctly anticipated. For example, 
nuclear conflict during the Cold War led to stability because both the U.S. and Russia 
correctly assessed the costs of going to war as prohibitively high. This is in stark contrast 
to the situation during Operation Iraqi Freedom, where the U.S. underestimated the costs 
of conflict with Iraq and the Iraqis incorrectly assessed it to be high. This miscalculation 


was disastrous for the Iraqis, but damaging for both sides (Haass, 2009). 


Comprehending the mathematical approaches used by potential adversaries helps 


decision-makers understand where risk miscalculations (by either side) may take place. 


This provides leadership with potential “inside knowledge” as to how an adversary makes 
decisions and prepares their contingency plans. Most importantly this knowledge 
highlights where likely missteps may take place, both to the advantage and disadvantage 


of a belligerent. 


B. METHODOLOGY 


The United States has a tradition of military analysis using a federated suite of 
models. These are not the only methods of modeling military problems. We consider the 
application and implications of foreign modeling approaches. The particular alternate 


technique we focus on is “Fuzzy Comprehensive Evaluation” (FCE). 


FCE makes use of fuzzy mathematics, alone and in partnership with the Analytic 
Hierarchy Process (AHP), to inform strategic and operational decisions. In 1965, Lotfi 
A. Zadeh introduced fuzzy mathematics as a way to capture uncertainty, particularly 
when assessing decisions with qualitative factors. Fuzzy mathematics has not gained 
widespread use in the United States, because other methods for dealing with uncertainty 
are preferred. AHP works by organizing all aspects or factors of a decision problem and 
providing a perspective of importance of the components of the decision. Fuzzy math 
and the AHP collaborate to form FCE, a decision-making tool used to evaluate competing 
decision alternatives or courses of action influenced by multiple factors. Eventually, the 
FCE determines which alternative is ‘best’ at meeting the main objective or decision 


question. Chapter II explains the FCE methodology. 


FCE methods are foreign. Their applications have been found in Chinese 
research, industry, systems engineering, as well as military applications (Li, Ma, & Liu, 
2004). Lotfi A. Zadeh, the creator of fuzzy logic, noted in a 1994 interview with 
Azerbaijan International how the support for fuzzy logic in China is beginning to 
increase since the loss of funding after events of Tiananmen Square and is “likely to grow 
substantially” (Blair, 1994, p. 47). Applications in evaluating naval tactical missile 
systems, attack helicopters, and modeling of U.S. overseas logistics operations are among 
the few studied by Chinese analysts (Cheng, 1996; Cheng, Yang, & Hwang, 1999; Mu, 
Guo, Niu, & Jia, 2013). During the interview, Professor Zadeh also highlighted 


substantial Japanese work in consumer products (i.e., appliances and electronics) using 


fuzzy logic. The Japanese are the top practitioners of fuzzy-logic based systems and 
products since the 1980s, according to Dr. Zadeh. With such success of several fuzzy 
logic based products in the east, a revival in using fuzzy methods began in the U.S. by the 
early 90s (Blair, 1994). As a result of the revival, the U.S. has considered applying fuzzy 
methods to military tracking systems (Smith, 1995) and neural networks for anomaly 
detection in military aircraft (Brotherton & Johnson, 2001). Today, as reported by the 
international journal Advances in Fuzzy Systems (2013), as of March 4, 2013, the U.S. 
has 16,898 patent applications and patents issued related to fuzzy logic. This is in stark 
contrast to the 7,149 patent applications and patents issued to fuzzy logic related work in 
Japan (Singh et al., 2013). But even though fuzzy analysis has gained greater acceptance 


in the U.S., the academic community still heavily scrutinizes this approach. 


Cy OBJECTIVE 


The goal of this thesis is to introduce the FCE methodology and research and 
analyze its vulnerabilities. We conduct an analysis of FCE vulnerabilities in a combat 
logistics resupply scenario and in a study of intentionally biased inputs. The goal is to 
meet demands at the lowest operating cost of the MOTs. This thesis accomplishes three 


research goals: 
1. It critiques the application of fuzzy math and AHP in FCE. 


2. It finds differences between the FCE approach and current U.S. approaches 


when applied to military decision support processes. 


3. It evaluates the implications of FCE results when there is a high variance in 


expert opinions. 


We examine the reasons why the results of optimization and FCE diverge. The 
knowledge gained is useful for discovering weaknesses in the FCE results, as well as 


understanding areas of strategic miscalculations when FCE is employed. 


Chapter II introduces the detailed process behind FCE by dissecting and 
explaining the method in its constituent components. Chapter II] compares and contrasts 
the FCE method with an optimization model in choosing the most cost effective logistics 


platform for a combat logistics resupply scenario. Chapter IV reviews results and 


findings from Chapter III and presents a new method to test the effect of bias in expert 
opinions on FCE results. Last, Chapter V concludes with insights and suggestions for 


applying the formulation and methods developed in Chapter IV. 


I. THE FUZZY COMPREHENSIVE ANALYSIS (FCE) METHOD 


FCE identifies its roots in two mathematical approaches, Fuzzy logic and AHP. 
Sections A and B describe the main components while Section C combines them in a 


discussion of FCE step by step. 


A. FUZZY LOGIC 


In 1965, Lotfi A. Zadeh, a professor at UC Berkeley, revolutionized the idea of 
set theory and conventional logic by introducing fuzzy logic in his paper “Fuzzy Sets.” 
Fuzzy logic is a form of probabilistic logic that deals with approximate inferences or 
ambiguous data (qualitative data) versus fixed or exact reasoning (binary yes/no choices 
and quantitative data) associated with conventional set theory. Professor Zadeh (personal 
communication, May 9, 2011) explained fuzzy logic as a, “logic of classes which do not 
have sharply defined boundaries.” The concept behind fuzzy set theory involves a 
determination of whether an object, number, or quality is a “member” and the extent or 
“degree of truth” (degree of confidence) that it is within a set (Zadeh, 1965). The degree 


of truth is a value between 0 and 1, rather than only 0 or | as in classical logic. 


The pejorative nature of the word “fuzzy” did not lend itself well in U.S. and 
English-speaking academic circles when it initially debuted in 1965. Additionally, its 
non-binary and imprecise way of thinking was criticized. There were some who did not 
believe in the partialities of truth presented by fuzzy logic or the use of “linguistic 
variables” (variables whose values are words) based on individual perception. Zadeh 
(personal communication, May 9, 2011) said that it is possible to compute with words, 
“you can add small to large... multiply them, etc.” because it can be “useful to use words 
instead of just numbers.” “Fuzzy Thinking” is a kind of scientific permissiveness based 
on imprecise results and subjective outcomes. However, some find this property 
especially useful when dealing with ambiguity in responses or situations involving 


imprecision. An example of ambiguity is when a statement is neither “true” nor “false”. 


Conventional set theory does not capture ambiguities of reality well. For 
example, conventional theory can only classify the lethality of a missile as either lethal or 


not. However, any competent decision-maker would want more information. More 


statistics on the degree of lethality would be valuable. Or if the missile is not lethal, how 
would this compare to another non-lethal missile? Clearly, both should have varying 
degrees of non-lethality. Problems such as this, where approximate classifications from 
subject matter experts provide better clarity to undefined phenomenon, is where fuzzy set 
theory lends itself. Fuzzy logic focuses on the subjectivity and imprecision found in 
human judgment. Stochastic methods and statistical analysis fall short in adequately 
handling the subjectivity and imprecisions of decision support processes. Fuzzy data can 
provide accurate insight by capturing the partialities associated with real human 


knowledge in decision-making (Yeh, Deng, & Pan, 1999). 


Fuzzy math differs from conventional math in the same manner as fuzzy logic is 
to conventional logic. In the missile example, fuzzy math does not categorize a missile 
as lethal with True or False (1 or 0). Instead fuzzy math categorizes a missile’s lethality 
with varying degrees of confidence or ‘degrees of truth’. For instance, if a value of 0.00 
is considered False and 1.00 is considered True, then a statement saying, “Missile A is 
lethal.” can have a truth or confidence value of 0.8 and false value of 0.2. Moreover, the 
concept of a fuzzy set is important in understanding applications of fuzzy math. A 
conventional set is merely a collection of numbers or objects. On the contrary, a fuzzy 
set is a list of values, which describe to what degree an object belongs to a variety of 


characteristics or criteria. 


B. ANALYTICAL HIERARCHY PROCESS 


The weighting of criteria and sub-criteria greatly affects the accuracy of fuzzy 
evaluation results (Li, Jiang, Li, & Mu, 2011). Generally a methodical evaluation process 
that can properly handle the prioritization of evaluation elements for a decision objective 
is invaluable. AHP examines the relative importance of each criterion and determines 
their relative weighting. There does not seem to be a unified method for determining 
weights for criteria. In practice, various methods are used. Some interesting ways for 
weighting are entropy weights, AHP, and even direct weighting from the decision-maker 
(Yan, Zhang, Zhang, & Wu, 2008; Yeh et al., 1999). The concept of entropy weights 


involve weights established based on the probability a realization will occur. Thus, the 


higher the probability a realization will occur, the higher its entropy weight will be. 


However, this thesis focuses on AHP and direct weighting from the decision-maker. 


AHP has been widely used in solving many complicated decision-making 
problems (Cheng, 1996; Cheng, Yang, & Hwang, 1999; Dagdeviren, Yavuz, & Kuling, 
2009). In the 1970s, Thomas L. Saaty developed AHP and explained its concepts in the 
following way. 

A basic premise of AHP is its reliance on the concept that much of what 

we consider to be “knowledge” actually pertains to our instinctive sense of 

the way things are. This would seem to agree with Descartes’ position that 

the mind itself is the first knowable principle. The AHP therefore takes as 

its premise the idea that it is our conception of reality that is crucial and 


not our conventional representations of reality by such means as statistics, 
etc. (Saaty, 1988, p. 110) 


AHP is a type of systematic evaluation model called a multiple criteria decision-making 
(MCDM) method. There are different types of MCDM processes, but all handle 
problems of subjectivity, ambiguity, and uncertainty involved in a multi-criteria selection 
problem (Dagdeviren et al., 2009). AHP addresses the relative importance of each 
criterion over another involved in a decision problem with the purpose of determining 
criteria weights. AAHP decomposes a problem, process, or course of action into the 
important factors that describe and form the overall evaluation. Criteria are decomposed 
into sub-criteria and the decomposition continues until a set of factors is established that 
can be directly evaluated. The evaluation set can consist of qualitative criterion values 
(i.e., lethality, reliability, attractiveness, stealth) requiring subject matter expert opinions 
to determine the relative importance of each to one another. Additionally, the evaluation 
set can contain quantitative (fuel economy, speed, cost, effective ranges, etc.) criterion 
values taken from test and evaluation sources or measured performance data. Therefore, 
AHP permits integration of intangible qualitative criteria alongside tangible quantitative 
criteria (Badri, 2001). Ultimately, AHP reconciles decision-maker’s or subject matter 
experts’ preferences (subjective inputs) for criteria and identifies the reliability of the 


preferences. 


AHP consists of three steps: first, determine a set of evaluation elements or 


criteria and organize them into a hierarchy tree for analysis; second, assess pairwise 


comparisons of elements at each level of the hierarchy to formulate criteria and sub- 
criteria weights; and third, synthesize the prioritizations of the alternatives through 


combining normalized max eigenvalues and criterion weights. 


Decision models are generally complex consisting of many decision elements or 
criteria with associated sub-criteria involved in the decision process. AHP’s hierarchy 
tree aids the decision-maker by organizing the complexities of a decision. Specifically, 
once the decision-maker establishes a criteria set, AHP organizes these contributing 
elements into an interrelated hierarchical structure called a hierarchy tree. Following the 
previous missile example, Figure 1 details a simple hierarchy tree for a scenario with 


three alternative missiles. 
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Sub-criteria 3.1 Sub-criteria 3.2 


Figure 1. Sample AHP hierarchy tree for alternative missile system selection problem 


The hierarchical decomposition of real events is a logical method for individuals 
to deal with the complexities of reality. The purpose of the hierarchy tree is to make 
criteria and sub-criteria evaluations easier across any given level to some or all criteria in 


upper levels. Additionally, the hierarchy tree ensures consideration is given to all aspects 


of a decision problem by mapping the problem’s criteria in the tree. As seen in Figure 1, 
the hierarchy tree is similar to a family tree with parent nodes represented by the main 
components involved in a given decision objective. Child nodes branch off parent nodes 
organizing potentially multiple levels of criteria and sub-criteria. The criteria and sub- 
criteria help evaluate each decision alternative or course of action. A typical AHP 
hierarchy has a minimum of three levels: overall goal, objective, or decision of the 
problem at the top of the tree, multiple criteria with sub-criteria defining the alternatives 


in the middle, and decision alternatives at the bottom (Dagdeviren et al., 2009). 


The initial estimates of criterion priorities (weights) are based on pairwise 
comparisons of criteria. Table 1, depicts a sample paired comparisons for the three sub- 


criteria of “Criteria 3.” 


Sub-Criteria 3.1 |Sub-Criteria 3.2 |Sub-Criteria 3.3 
Sub-Criteria 3.1 a ee) 


Sub-Criteria 3.2 
Sub-Criteria 3.3 i; 3 5 








Table 1. | Sample pair-wise comparisons of Criteria 3’s sub-criterion 


Subject matter expert opinions rank one criterion over another to create pairwise 
comparison matrices (similar to Table 1) at each level of the hierarchy tree. Table 1 
displays one subject matter expert’s opinion of how they feel each sub-criterion ranks in 
importance to the others. The values from the pairwise matrices come from Saaty’s 1-9 
scale in Table 2. Each value in the 1 through 9 scale has associated verbal descriptions. 
The pairwise matrix is read as follows. If the a criterion is evaluated with itself, they 
would be equally as important to each other and given the value one. Otherwise, 
referring to the criteria of the first entity (vertical criteria) and the second entity 
(horizontal criteria) for differing criteria would produce something different. The 
decision-maker determined the first entity, sub-criterion 3.1, is strongly more important 
than the second entity, sub-criterion 3.2 thus a value five was given. Understand that the 
values below the diagonal of 1’s are simply the reciprocal of its counterpart. So in the 


case of sub-criterion 3.1 being strongly more important than sub-criterion 3.2, conversely 


sub-criterion 3.2 is strongly less important than sub-criterion 3.1. However, as with any 


subjective input, potentially inconsistent judgments from experts may occur. 


SS 


The first entity is very strongly more important than the second 
The first entity is absolutely more important than the second 
2, 4, 6, 8 Intermediate or compromise values between the above categories 


Reciprocals are used to represent inverse relationships, e.g. 1/3 
1/3, 1/5, 1/7, 1/9 
indicates that the second entity is weakly more important than the /irst. 





Table 2. Verbal description of 1 to 9 scale (from Zuiliang. et al., 1993) 


Fortunately the AHP provides an approach for improving consistency. A 
“consistency ratio” (CR) effectively reduces the introduction of bias from decision- 
makers (Badri, 2001). If the consistency ratio is less than or equal to 0.10, then the 
subjective inputs that formulated the weights for criteria are considered appropriately 
valid or “adequately consistent.” Equally, if the CR is greater than 0.10, then this means 


that the pairwise comparisons must be repeated until a consistent matrix is obtained. 


The maximum lambda, or maximum eigenvalue, of a paired comparison matrix is 
needed to solve for the consistency index (CI). The CI is calculated from Equation (1), 
where n is the number of criteria: 

Anna —) 
a (1) 


The CI is used to calculate the CR. The CR dctaeniines if a paired comparison matrix is 


Cl= 


sufficiently consistent to evaluate criteria weights when CR is less than or equal to 0.1. 


Equation (2) details how to calculate the CR: 
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CI 


R= 
R(n) (2) 
where R(n), is the random index. It is a known value dependent on the size of the matrix. 





For example, R(n) value for a 6x6 matrix is 1.24 (Hahn, n.d.). 


In addition to ensuring the consistency of a decision matrix, AHP has two modes 
for synthesizing the prioritization of alternatives: the distributive and ideal modes. The 
distributive mode allows for violations in the deductive process of logic (Saaty, 1988). 
Consider a decision-maker who chooses Missile A over Missile B and Missile C. 
However, using the distributive mode the “inclusion of irrelevant alternatives,” or decoy 
alternatives, can cause rank reversal when new alternatives are presented in the presence 
of old alternatives. This means that by presenting an irrelevant alternative, Missile D, the 
decision-maker may now choose Missile B. The distributive mode is also useful when 
there is dependence on the number of alternatives present. The ideal mode is the form of 
synthesizing priorities where rank reversal is not allowed. This is because the ideal mode 
preserves rank within its calculations. The ideal mode method divides the normalized 
values of the alternatives for each criterion by the value of the highest rated alternative 
before multiplying these priorities with local weights of each criterion. In this way, the 
addition of any new alternative that may dominate everywhere cannot cause rank reversal 


amongst existing alternatives (Saaty, 2001). 


Note that FCE replaces the above third step with a fuzzy composition method 
(explained in detail later) to synthesize or produce fuzzy prioritizations of the 
alternatives. Thus, in FCE, the AHP is only used to organize elements of a decision into 


a hierarchy tree and formulate criteria and sub-criteria weights. 


OF FCE 


Fuzzy logic and AHP combine to construct the basis for the fuzzy comprehensive 
evaluation method used in this thesis. FCE is a five-step process. The first steps are 
based on AHP and the rest incorporates a fuzzy composition method that aggregates 
criteria weights and grade scores to produce an overall evaluation score for each 


alternative. The highest ranked alternative is provided to the decision-maker from the 
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resulting fuzzy evaluation. The following is an overview describing the five steps. 


Chapter III provides a detailed exercise of the FCE method. 


1. Determine the Criteria Set and Develop the Evaluation Hierarchy 


A decision question of “Which missile system is best to employ against surface 
ships?” can be mapped with corresponding criteria and sub-criteria as seen in Figure 1, 


the sample hierarchy of alternative missiles. 


2. AHP and Determining Element Weights 


AHP estimates the additive utility weight for the criteria that best match the initial 
information provided by the decision-maker. AHP uses decision-maker or subject matter 
expert panel’s opinions to perform paired judgments starting at sub-criteria and then 
moving up to corresponding criteria on concurrent levels to determine sub-criteria and 
criteria weights for the whole decision problem. The pairwise judgments form 


prioritizations of criteria and formulate criteria weights. 


3. Establish Linguistic Grade Set and Evaluation Set 


This step results in the development of a method for how each alternative should 
be evaluated for each criteria starting from the lowest level of the hierarchy. If expert 
opinions are incorporated due to qualitative criteria, then a linguistic grade scale is 
established. The result of this step forms the qualitative linguistic grade scale (excellent, 
good, fair, poor) and a way of mapping these qualitative scores onto a quantitative scale 
(excellent = 4, good = 3, fair = 2, poor = 1). The linguistic grades can be a five-point 


scale or any amount depending on what a decision-maker needs. 


4. Evaluate the Alternatives’ Fuzzy Judgment Matrix 


Next determine the linguistic score for which an alternative satisfies a particular 
criterion. For example, the degree Missile A satisfies Criterion 1 may be judged as 


> 


“excellent” while the degree Missile B satisfies Criterion | is “poor.” This computation 
begins from the lowest level sub-criteria up to the highest criteria level creating a 
“subjective assessment matrix” for each alternative. An m x n subjective assessment 


matrix results where m is the number of criteria and n is the linguistic grade category 
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(i.e., excellent, good, fair, and poor). Linguistic grades are useful in expressing the 
vagueness and partiality in expert opinions. Linguistic grade values facilitate the making 


of qualitative assessments. 


Scaled probability matrices called fuzzy judgment matrices are calculated from 
the subjective assessment matrices. The scaled fuzzy judgment matrices are based on the 
total number of experts, N, who provided assessments. The fuzzy judgment matrix 
displays the scaled degree of membership within a linguistic grade that an alternative 
demonstrates in terms of each criterion. Refer to Chapter III, Section E for a detailed 


discussion. 


3: Calculate Overall Synthetic Result 


The overall optimal fuzzy evaluation is determined for each alternative. The 
overall fuzzy result is calculated by multiplying fuzzy judgment matrices for each 
alternative with criteria weight vectors. Furthermore, methods to select the “best” 


alternative when two are within the same category are applied to resolve ties. 
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Hil. FCE IN MILITARY LOGISTICS 


A notional logistics resupply example from the Joint Campaign Analysis (JCA) 
course at the Naval Postgraduate School provides the framework to study the 
implications of using FCE versus a mixed integer linear program (MILP). The MILP 
was originally implemented to solve a multi-criteria decision problem in the JCA course. 
We demonstrate the FCE method on the same scenario to study the FCE approach and 


compare its results to the MILP. This notional example is for research purposes only. 


The following chapter first presents comparison of the U.S. favored approach in 
optimization to the foreign preferred FCE. We next discuss the background and 
modeling assumptions of the motivating scenario. We then describe of the MILP 
implementation from the original study. Followed by a thorough application of the FCE 
method performed on a combat logistics scenario. The chapter concludes with insights 


and remarks on how FCE performed and a comparison of results from both methods. 


A. OPTIMIZATION VS FCE 


In the following study, the MILP solves the logistic problem through an 
optimization approach. The optimal solution is determined through the minimization of 
an objective function centered on the operating costs of each alternative. Additionally, 
the objective function is constrained by the number of available logistic means of 
transportation (MOT), the number of trips each can make within a 50-hour cycle, and 


cargo limitations to meet the demand of warships. 


The MILP implies a requirement for certain preciseness in the data. In order to 
accomplish this, a stochastic optimization is most appropriate. In this situation, one or 
more input parameters are subject to randomness in order to capture the uncertainty and 
imprecisions of real-life phenomena. The focus of this thesis is not on the effectiveness 
of preferred U.S. methodologies. Therefore, we did not pursue a stochastic optimization 
program. The intention of highlighting a stochastic optimization program is to 
demonstrate the weaknesses and strengths of optimization for comparison to the 
alternative, FCE. Knowledge of the parameters’ distribution is also required to simulate 


the randomness of a parameter in optimization. This makes the optimization approach 
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data intensive because larger sample sizes of data provide better estimates of the data 


distribution. 


FCE solves ambiguous and imprecise information in the combat logistics problem 
through human knowledge for each logistic ship. Explicitly, FCE takes subject matter 
expert opinions as input data and through the FCE transforms them into quantitative 
output. FCE and optimization seem to be diametrically opposed methods. One method 
approaches problems subjectively (FCE) while the other approaches the same problem 
objectively (MILP). In terms of providing policy analysis for DoD purposes, FCE would 
prosecute the “best” policy alternatives with influences from policy-makers. However, 
optimization would determine the optimal solution subject to performance data and 
constraints of alternative policies. Selecting which methodology to employ depends on 


the area of interest and purpose of the user. 


B. SCENARIO 


The current concept of operations for the U.S. Navy’s combat logistics supply 
process is to employ large, slow moving, and minimally defended logistical ships such as 
the USNS Lewis and Clark Class auxiliary cargo (K)/ammunition (E) ship (T-AKE). 
Additionally, the Navy faces an inability to resupply missiles at sea because there are no 
resupply platforms or procedures for this capability. A potential solution to these 
limitations is implementing the concept of a Mobile Landing Platform (MLP) as a 
resupply point at sea. An MLP is essentially a small highly mobile barge that is capable 
of replenishing a Surface Action Group (SAG) with supplies and strike missiles. The 
objective of this scenario is determining the most cost effective naval combat logistic 
force (CLF) mix to resupply a “Repositionable Expeditionary Basing” concept where the 
MLP operates as a repositionable expeditionary base (REB). 


A “War at Sea” strategy is necessary to combat heightened hostilities from the 
Chinese with their claim to majority of the South China Sea (SCS). The “War at Sea” 
strategy implements a credible force structure with associated logistics, deployment 
strategy, and alliances to effectively deter Chinese aggression in the SCS. The 
repositionable basing concept merely encompasses an option for logistical solutions 


within the endeavors of the overall strategy. The overall goal is to determine which of 
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three logistics MOT are the most cost effective to employ in this scenario. Specifically, 
how do we minimize the cost associated with resupplying a SAG and Flotilla fleet during 
conflict? The Repositionable Expeditionary Basing concept initially provided a “Blue” 
optimized analysis for a U.S. logistics force supporting the ““War at Sea” strategy. We 
take this logistics analysis a step further by using Chinese analytical methods such as 


FCE to seek the “Red” solution in the following case study. 


C. MODELING ASSUMPTIONS 


REBs are flexible locations within a high-risk area (subject to attack) where 
logistic ships offload supplies and U.S. combat ships receive supplies. Figure 2 illustrates 
the proposed resupply process with the transit scheme between a transfer point and a 
single repositionable base. Any REB can be either close (SOONM) or far (between 
500NM and 1000NM) away from a transfer point node. But initially a T-AKE provides 
supplies to the model, at a transfer point. Transfer points are always reasonably in safe 
areas (port or prescribed friendly locations) clear from potential attacks. The transfer 
points are start nodes, where alternative logistic MOTs move supplies into the system. 
The alternative MOTs are the vehicles capable of transiting in and out of an area of threat 
or “risk” to get to a REB. All MOTs offload only two types of cargo: fuel and missiles. 
Each logistic MOT’s transit speed is the same (15knots) towards a REB (blue arc) in 
order to reduce their detection and blend in with surrounding merchant ships and local 
fishing vessels. However, once a logistical MOT unloads its supplies, the assumption is 
that they are detected and must expeditiously exit the ‘area of risk’ (red arc) at max speed 
(sprint speed) back to the transfer point to restart the cycle. Once the REB receives 
demanded supplies, the SAG ships or Flotilla fleet (green arrows) can resupply from 


them as needed. 
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T-AKE with Supplies Warships with Demand 


Full Load, Slow Transit 
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Empty load, Fast Transit 


Area of Safety Area of Risk 


Threat Demarcation 


Figure 2. Transit scheme between a transfer point and a single base. 


Figure 3 depicts the logistic network we analyze. A transfer point with alternative 
logistic ships on the left provide supplies to four possible REB locations on the right. 
Although Figure 3 illustrates one transfer point, multiple transfer points are possible 
subject to operational needs. The REB locations represent the demand nodes for the 
logistics network. Their locations are arbitrary. However, two REBs must be within 
500NM of a transfer point (close) and two REBs between 500NM-1000NM (far). The 
location requirements for the REBs within an Area of Responsibility (AOR) are to 
provide redundant and flexible resupply points for combat operations. Conceivably more 


REBs can be placed “close” or “far.” 


The logistics network evaluates the performance of combinations of logistic MOT 
in determining the most cost effective logistics force mix. The thicker lines symbolize 
higher supply capacities because more trips can occur on them. The lighter lines depict 


lower supply capacities due to less number of trips made to further REBs. 
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Figure 3. Disposition of possible REB locations to a transfer point. 


The assets requiring supplies are SAG, consisting of three destroyers, and a 
Flotilla fleet of 34 small combatant ships. Notional fuel and missile requirements 
required for one days’ conflict are used to estimate demand. Parameters for each logistic 
alternative were researched to determine capabilities. The three alternatives investigated 
for logistic resupply are the LCS-2 (Independence variant Littoral Combat Ship) and two 
modified versions of the Joint High-Speed Vessel (JHSV). JHSV refers to the version 
with lower fuel capability and higher missile capability. Combat Logistic Ship 5000tons 
(CLS-5000) is the modified JHSV with higher fuel capacity. Since the CLS-5000 is a 
concept in development, it provides potential design modifications to current JHSV 
capabilities and missions. Additionally, both the LCS-2 and JHSV are the latest ships 
employed by the U.S. Navy. Their missions and capabilities are still in question. Thus, 
this study was meant to explore potential logistics employment opportunities for both 
models. All unit parameters are defined for a 50-hour time step. Table 3 displays each 


alternative’s parameters for this study. 
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JHSV 


, 


Operating Cost S 27,770,833.33 | S 21,500,000.00 | S 58,700,000.0 


0.75 


ee) 
Table 3. Alternative logistical MOT parameters. 





According to Austal, the manufacturers’ specifications for sprint speeds and 
missile capacities for the LCS-2 and JHSV are detailed in Table 3 (Joint High-Speed 
Vessel & Littoral Combat Ship, n.d.). Transit and sprint speeds are combined into “sprint 
speed” within FCE analysis because since the values for transit speed are the same across 
all MOTs, its inclusion is irrelevant. On the other hand, transit speed performs an 
important function in the optimization model. Transit and sprint speeds capture the time 
a logistic MOT travels slowly to a REB and rapidly back to the transfer point after 
delivering supplies within 50 hours. Missile capacities were based off maximum payload 
capability of each vessel. Fuel tanker capacities equal 20% of the current maximum fuel 
capability of each alternative. Operating costs for the LCS-2 and JHSV are 10% of total 
unit program costs taken from the U.S. Government Accountability Office’s (GAO) 
Assessments of Selected Weapons Programs review (Sullivan, 2013). However, the CLS- 
5000’s operating costs are calculated from the JHSV’s operating cost aggregated due to 
upgrades of JHSV to CLS-5000 capabilities. 


In maintaining simplicity and thesis goals, modeling lethality and reliability (how 
many hits can an alternative sustain) were not the focus of the optimization model. 
However, the opportunity to explore these effects of increased defensive capabilities or 
reliability of an alternative to complete its mission (maintenance schedules, operating 
readiness of crew, etc.) is provided within the “probability of kill” (P(Kill)) and reliability 
criteria and left for further study. Both parameters were not incorporated into the MILP 
because its inclusion would have taken the MILP beyond the scope of this case study and 
made comparisons to FCE difficult. However, FCE’s incorporation of them was 
appropriate in capturing the factors affecting each alternative’s rankings. The P(Kill) 


values found in Table 3 are provided based on the unclassified defensive capabilities of 
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each ship. The reliability values are estimated from author experience given that 


reliability is fairly variable in the real world. 


D. OPTIMIZATION MODEL 


In answering the research question of determining the most cost effective logistics 
force mix to employ in this scenario, we formulate a MILP, implement it in MS Excel, 


and solve it using the default MS Excel MILP Solver. 


The following definitions and equations explain the optimization model as we 


attempt to satisfy combat operational demands while minimizing operating cost. 


1, Sets: 

i€ [={CLS-5000, JHSV, LCS-2} vessel type 

J €J ={fuel, missiles} cargo type 

k € K ={500NM, 1000NM} distance REB is located 
r € R={Close, Close to Close, Far} route type 


2; Data [units]: 


re = operating cost of vessel type i [3] 


= capacity of each vessel type i for cargo j [ST/trip] 


bes = trips vessel 7 makes to distance & on route r [trips/vessel] 
mM, = number of bases at distance k [bases] 
a : = demand for cargo j at distance k [ST/base] 
J) 
S, = number of available vessels of type i [vessel] 
a: Decision variables: 
X te Integer (= 0) Number of vessel type i used on route r 
4. Formulation: 
min) eX, (3) 
ivr iel reR 
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Sh. Dy ait Sma. Vik (4) 


iel reR 


ys Vi (5) 


reR 


X,,, 20 and integer Vir (6) 


5, Discussion 


Equation (3) is the objective function. It measures the cost of employing vessels. 
Table 4 details the specified values for the parameters of each alternative MOT for the 
MILP. The cycle length of 50 hours is based off the time to complete far trips. The time 
to complete round trips to close or far bases are determined from each alternative’s transit 
speed to a REB and sprint speeds back to a transfer point. Only two close and two far 
REBs are modeled totaling four within an area of operation (AO). The total wartime 
missile and fuel demands, measured in short tons (ST) per hour, are required for each 
close and far base. This supports the redundancy of the system. If three out of the four 
REBs are eliminated, the remaining REB is capable of supplying all combat ships within 
the AO. Missile demands are hypothetically calculated for the SAG and Flotilla fleet. 
The fuel demands are calculated with the assumption that the SAG and Flotilla group 
always maintain 85% fuel capability. Therefore, they require about 1000ST of fuel to 


maintain operability according to operating procedures. 








Constraints LCs-2 

Cycle length 50 hours 50 hours 50 hours 
Total_Number_Close REBs to resupply m_k 2 2 
Total_Number_Far REBs to resupply 2 
Total Close base_Wartime_Missile_Demand [ST] 1110 1110 
Total Far base_Wartime_Fuel_Demand [ST] D_jk 2000 2000 2000 
Number_Close_Trips 2.1 2.2 
Number_Far_Trips 1.1 1.1 
Available number 15 16 








Table 4. | Parameter values for optimization model 


The possible routes a vessel can travel to provide supplies are as follows: two 


trips to close REBs or one trip to a far REB within 50 hours. The last route of only 
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delivering to one close REB is to meet any leftover demand. Therefore, 7;,, assumes a 


value of about one or two depending on whether a close or far trip is taken. The trip 
index, k, accounts for the worst-case scenario when the MILP selects an alternative to go 
close or far. For example, if a REB is 600NM away from a transfer point, the MILP 


assumes the worst case and calculates the length of this trip with a 1000NM distance. 


This means that 7; ,,, can only make a maximum of one trip to this REB and assumes the 


value of one. Lastly, the available number of alternative ships is based on projected 
productions of each alternative by 2025. This was a stipulation from the JCA course 


where this example originated. 


The first constraint set, Equation (4), ensures demand is satisfied. They allow for 
oversupply of the REBs without penalty. The second constraint set, Equation (5) limits 
the number of available vessels. Lastly, the decision variable is non-negative and integer 


as seen in Equation (6). 


6. MILP Solution 


The MILP found the most cost effective logistical force to resupply U.S. combat 
forces engaged in an Area of Operation (AO) in the S. China Sea was nine Combat 
Logistic Ships 5000tons (CLS-5000). Three CLS-5000 vessels were required to fulfill 
the demand of “close” REBs and six CLS-5000 fulfilled the demands for “far” REBs. 
There was no better combination of a CLF mix of logistics ships. The CLS-5000 
provided the optimal solution in terms of cost and cargo capacity to resupply. Yet to 
verify the fidelity of these results and provide a ranking of the alternatives, the 
optimization model was run without the optimal choice of CLS-5000 as an option. The 
results were eight LCS-2 (seven required to supply “far” bases and one for “close” bases) 
and 15 JHSV’s (eight to supply “far” bases and seven to supply “close” bases) to fulfill 
what nine CLS-5000s accomplished alone. These results indicate that the ranking of the 
alternative methods of logistical transportation are 1) CLS-5000 2) JHSV 3) LCS-2 based 
on the number of ships employed to meet demand. The lower number of ships MILP 


selected equates to higher cost. 
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E. FCE MODEL 


In this section we examine an FCE approach to the combat logistic resupply 
scenario. FCE is useful when human knowledge is necessary and human evaluations are 
needed due to the presence of incomplete or vague information. Each alternative MOT 
possesses differing capabilities or factors that render it useful for employment in a 
logistics mission. An advantage is that we already know most of the factors involved in 
this scenario from the MILP model. However, it is difficult to systematically evaluate 
how each alternative’s capabilities meet a decision-maker’s objective without expert 
knowledge and a method for analyzing performance data. FCE is a method that can 
suitably prioritize the alternatives to best meet a mission’s intent. The following are the 


steps for this qualitative assessment. 


1. Determine Criteria Set and Develop Evaluation Hierarchy 


We use the following six criteria to evaluate the alternatives: fuel capacity, 
missile capacity, sprint speed, probability of kill, replacement cost, and reliability, are 
used to evaluate the alternatives. These elements make up the set, FE, that are important in 


describing the qualities needed in a logistical MOT. In Equation (7), e, represents the as 


evaluation element in the set E, 
E={@,, €, ---> €,}- (7) 
A hierarchy tree is formed organizing the criteria for each alternative. Figure 4 
illustrates this study’s hierarchy as it relates the criteria to the alternatives evaluated. At 
the top of the hierarchy is the objective and focus of this study. The second level consists 
of the six criteria. The last level displays the three alternatives and their relationship with 


each criterion. 
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Which vessel is most 
cost effective to 
employ? 







FUEL 
CAPACITY 


CLS5000 





Figure 4. Logistic resupply hierarchy 


De AHP and Determine Element Weights 


The second step is a comparative judgment of the alternatives based on each 
criterion by subject matter experts. This begins the determination of importance of each 
criterion among the rest within a level of the hierarchy. AHP uses Saaty’s standardized 


scale from | to 9 for the multiple pairwise comparisons. 


Through pairwise comparisons of each criterion (with exception of transit speed) 
against the rest, appropriate weights for each criterion are determined. Table 5 depicts 
potential values for the paired comparisons based on our expert judgment of the middle 
level of criteria illustrated in the logistic resupply scenario’s hierarchy tree. These values 
would be determined by the decision-maker choosing the level of importance (from 
Saaty’s 1-9 scale) for a criterion to another. The same criterion would be equally as 
important to itself and given the value one when compared to itself on the pairwise 
comparison matrix. Otherwise, referring to the criteria of the first entity (vertical criteria) 
and the second entity (horizontal criteria) for missile capacity would produce the 
following. In this case, the decision-maker determined the first entity, fuel capacity, is 
less than weakly more important than the second entity of missile capacity (highlighted 
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gray in Table 5). On the other hand, values below the diagonal of 1’s are simply the 
reciprocal of its counterpart value. So if fuel capacity is less than weakly more important 
than missile capacity, then missile capacity is more than weakly more important than fuel 
capacity (highlighted yellow in Table 5). Because this analysis focuses on the cost of 
operation, cost is generally more than strongly more important that any other criterion. 
The values in Table 5 were logically generated from our knowledge, research and 
experience of each alternative while we acted as the decision-maker. The purpose for 
generating the pairwise matrix, by acting as the decision-maker, is to incorporate real 
human knowledge and evaluations in preparation for the simulation robustness check 


discussed in Chapter IV. 


ee) 


1 ) 





Table 5. Pairwise comparisons of criteria with fractions converted to decimals 


After the pairwise comparisons are completed, the eigenvectors of the matrix are 
calculated. The largest real eigenvalues of each row’s eigenvector form the max 
eigenvector of the matrix. Then, the normalized values of the maximum eigenvector 
formulate the weights in AHP. However, a second and equivalent procedure commonly 
used is calculating the geometric mean of each row of a pairwise decision matrix. 
Equation (8) describes how to calculate the weights through geometric means for the set 


of G. P is the n x n pairwise comparison matrix (Table 5) formed from subjective inputs: 
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Pn _ Pos 


G=1 8); Boy 5 By) 





2, = 4D. X Py XX Dy, (8) 


and each row of P calculates a value for g,. 


Once all the g, values are calculated for all 7 in 1...m, they are normalized 


forming the criteria weights. Equation (9) demonstrates how the criterion weight for fuel 


capacity is calculated: 


g, =N1xX2x4x3x0.167x4 
g, =1.587 


1.587 


aE 
yg, 
i=l 





1 


(9) 
for all g,. 


Table 6, displays the resulting weight vector for each criterion to the overall 


objective given expert inputs. 


weg | oar | om ‘| oo | oo7 | 057 | 004 | 


Table 6. Weight vector 


The set of weights make up the weight vector, W, found in Equation (10): 
W ={W,, Wy, -0+) Wat (10) 
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3: Establish Linguistic Grade Set and Evaluation Set 


Four linguistic assessment terms were chosen for this study’s evaluation set: 
excellent(v,), good(v,), fair(v,), and poor(v,). Thus, Equation (11), V, is the linguistic 
assessment grade vector used to score each alternative. The next section clarifies the role 
of V: 

V={), V,, V5 V,t- (11) 


4. Evaluate the Alternatives’ Fuzzy Judgment Matrix 


Evaluating each alternative’s membership in a linguistic grade for a certain 
criterion forms the subjective assessment matrices. These evaluations begin from the 
lowest level child nodes of the evaluation hierarchy and results are aggregated up towards 
parent nodes as described in Chapter II. For this study, we provide the subjective 
judgments that grade the alternatives based on each criterion by pretending to be multiple 
(20) experts. Of the 20 votes, 10 votes were naive opinions (inexperienced judgments) 
and the other 10 votes were subject matter expert judgments. The sources for expert 
opinions are gathered from studying personal experiences from LCS-2 operators and 
through research of the alternatives. Table 7 is the subjective assessment matrix as a 


result of the 20 evaluations. 
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Criteria/Grade 


Criteria/Grade 


Criteria/Grade 





Table 7. (Notional) Subjective assessment matrices 


In Equation (12), for every e,, f, represents the degree of membership on e. tov,, 


(i =excellent, good, fair, poor), based on subjective votes: 


i 
ay (12) 


In Equation (12), 1, is the number of e, € v,(n, are the values found in Table 7); 
Nis the total number of subjective votes (N =20). 


Equation (13) is the scaled subjective assessment matrix of an alternative called a 


fuzzy judgment matrix, F, of element e, , on grade value v,, created from every f,;: 


dye Dy 
Fal io: (13) 


i _ i 
Table 8 is the fuzzy judgment matrices for the LCS-2, JHSV and CLS-5000. 
Essentially, it is a rescaled from the subjective assessment matrix based on the number of 


subjective votes for each e,, on grade value v, out of 20 votes. 
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Criteria/Grade | Row Sum | Logistic Ship 


Criteria/Grade 


Criteria/Grade 


b 


p06 | 015 _| 
a a er 
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Table 8. Fuzzy judgment matrices 


5. Calculate Overall Synthetic Result 


Equation (14) is the fuzzy composition method used to calculate the overall 
synthetic results. The vector of criteria weights, W, are multiplied to the fuzzy judgment 


matrix of each alternative: 


Ti oe oe 
Yew XP ={Wij Wa ws Wet) tO. SE Oo ee 2 4) 
fo << Ff, 

The Y is a fuzzy vector representing all evaluation elements’ contributions and the 
degree of membership of each alternative for a grade value. Table 9 displays the 
distribution of the overall synthetic results for each alternative. Additionally, the modes 
of the overall fuzzy results of each alternative are highlighted in yellow. Note that for the 
LCS-2 there is almost maximum equal membership within the fair and poor grade values. 
This presents areas of miscalculation especially when an alternative has higher 
membership in a higher-grade category and just a little less but similar membership in the 
lower categories. Only the mode is taken as the overall result. No consideration is given 
to the distribution of grade scores. The “Insights and Conclusions” section of this chapter 


provides more discussion on this issue. 
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Table 9. Overall synthetic evaluation results 


The CLS-5000, modified JHSV won as the best candidate to supply warships 
engaged in the S. China Sea. The CLS-5000 had the highest quality score in the excellent 
grade. The remaining rankings are as follows: 2) JHSV 3) LCS-2. The MILP model also 


resulted in the same rankings. 


F. INSIGHTS AND CONCLUSIONS 


Both FCE and the optimization program produced the same results of selecting 
the CLS-5000 as the most cost efficient logistics vehicle. FCE handled the imprecision 
and subjectivity of reality well. FCE captured the same results as a robust mathematical 
MILP, but through the use of subject matter expert judgments. Yet, it is not free of 
weaknesses. FCE models the imprecision and subjectivity but does not provide 
techniques to contend with bias. Due to the subjectivity of the FCE method, the results 
are only as good as the expert inputs. Moreover, decision-makers can artificially weight 
inputs from senior experts over junior experts simply due to seniority, not due to who is 
more knowledgeable. Thus, for results to be more accurate, it is necessary to obtain 
opinions from a sizeable pool of experts, V (number of experts), in order to deal with the 


inherent bias of reality. Chapter IV focuses more on this issue in FCE. 


Another pitfall of FCE is how the overall results are determined. FCE only 
considers the mode of results as the final score for an alternative. FCE does not consider 
the distribution of each alternative’s resulting membership value across the linguistic 
grades. If an alternative has 51% membership in the excellent category and 49% in the 
poor, according to AHP the alternative is still considered excellent. The methods for 
breaking ties are problematic as well. During this study, situations arose when both 
methods for breaking ties (Max Degree of Membership or Ordered Weighted Averaging) 


provided contradicting results. Table 10 demonstrates the difficulty in separating the best 
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alternative amongst two alternatives evaluated within the same category. Unfortunately, 


no clear guidance is available to handle this. 








0.157 0.051 





0.1435 0.0515 








0.2905 








Table 10. Fuzzy evaluation with alternatives within the same category 


LCS-2 and CLS-5000 tie for having the highest-grade membership in “excellent.” 
Two methods are commonly used to break ties, “Max Degree of Membership” and 
“Ordered Weighted Average.” The Max Degree of Membership method takes the 
alternative with the highest degree of membership in a grade as the winner. Thus, the 
CLS-5000 is the winner according to Table 10. The Ordered Weighted Averaging 
method determines the better alternative by aggregating an alternative’s fuzzy vector with 
a value scale from | to 4, where excellent = 4, good = 3, fair = 2, poor = 1, as seen in 
Equation (15): 
l= {4, 3, 2, 1}. (15) 


Equation (16) produces the final ordered weighted average for each alternative, X: 


_ YX _ 1 Yas Yor Yad X44, 3, 2, DE 
4 4 


x (16) 


CLS-5000 wins again with an ordered weighted score of 0.79 versus 0.65 for LCS-2. 


Lastly, the ability of the decision-maker to provide direct weighting for criteria 
can be advantageous or disastrous. The criteria weighting can greatly affect the accuracy 
of the results. The resulting decisive alternative or course of action is sensitive to 
weighting changes of the criteria and sub-criteria. A slight change in the weight of a 
criterion can result in different decisions. Therefore, the accuracy of the criteria weights 


is paramount. 
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Chapter IV extends the logistics case study to investigate the effects of high 
variance in expert opinion. Specifically, the issue of expert bias is addressed by testing 


the robustness of expert opinions when a “rogue” expert is introduced. 
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IV. HIGH VARIANCE IN EXPERT INPUT STUDY 


In this chapter, we seek to determine how one “rogue” or adversarial expert can 
bias FCE results. Remember, FCE results are as useful as the subjective inputs given by 
experts. If an adversarial expert were capable of breaching enemy decision support 
processes, such as FCE, their goal would be to increase the miscalculations FCE 
provides. By inserting incorrect or inverse votes into the FCE system, we study how an 
adversarial expert can effectively bias and influence the results. We find that 
inexperienced or misinformed experts can have the same effect on the results as an 


adversarial expert. 


The goal of this study is to test and verify the robustness in FCE results in the 
presence of misleading information using the SCS logistic resupply scenario from 
Chapter III. We formulate a program using the R statistical language to evaluate the 
uncertainty of FCE results and test the effect of expert bias FCE rankings. The “R2B” 
(robustness to bias) tester simulates N (number of experts) subjective matrices from the 
original subjective assessment matrices of 20 notional experts (from Table 7) in Chapter 
Ill. Out of the N experts, a person’s subjective assessment matrix is simulated from an 
adversarial standpoint. The adversarial subjective judgment matrices, for each 
alternative, are generated from reversing a given set of “true” distributions of opinions. 
Specifically the “true” distributions’ probabilities are switched as follows: excellent with 
poor and fair with good. These “true” opinions come from the original subjective 
assessment matrices that we know produce the CLS-5000 as the best alternative used in 
Chapter III’s case study. Effectively R2B simulates expert opinions from a distribution 
of opinions and compares the results to the “real” results. After a few parameters are 
initialized and set, running the R2B tester provides users the number of experts required 
to maintain accurate results for the particular scenario they are studying. The following 
chapter discusses the R2B tester algorithm in detail and reviews some key findings from 


this study. 
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A. “R2B” FUNCTION 
1. Set/Initialize Parameters 


R2B is an R statistical function that requires four arguments (n, m, Nmax, and 


wts). Figure 5 displays the R2B function’s pseudo code. 








R2B=function(n,m,Nmax,wts) { 
Save_alt=matrix (0,nrow=Nmax,ncol=m, byrow=TRUE) 
for (N in 2 to Nmax) { 
for .(q in. 122.00) ¢ 
SME matrix<alt> = simSME(true_matrix,N) 
Normalize SME matrix<alt> 
Fuzzy matrix<alt> = FuzzyResults (wts,norm_ SME<alt>) 
Normalize Fuzzy matrices 
Save results as probabilities of the number of times an 
alt. lands in a linguistic category for each alternative 






































} 
} 
print (results) 


} 





Figure 5. R2B pseudo code 


The user must input the number of criteria (fuel capacity, sprint speed, cost, etc.), 
n, that the alternatives are evaluated upon. Input the number of linguistic categories, m, 
after establishing the linguistic grade values (excellent, good, etc.). Normally the 
decision-maker determines this. Currently m is set to four because the logistic scenario 
had four linguistic categories: excellent, good, fair, and poor. Nmax is a variable 
describing the maximum number of experts to evaluate. R2B has the default setting of 
Nmax equal to 10. This means that a maximum of 10 experts are evaluated. A user is 
able to find the effects of one rogue expert amongst any number of experts between two 
up to a maximum 10. This default setting reflects the difficulty in amassing large 
numbers of experts at once. However, dependent on the scenario presented to R2B, it is 
possible that more than 10 experts are required for conclusive results in the presence of 
one rogue expert. The last argument, ws, is the AHP criterion weights. Once the 
weights are determined by either AHP, either given directly from the decision-maker, or 


taken from a sample problem, set the wts parameter to those values. 
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2. Import “True” Fuzzy Subjective Assessment Matrices 


The user must first import the original, distribution of fuzzy subjective assessment 
opinions as matrices into the R statistics package. Although not direct inputs into R2B, 
these true matrices are coded into the R2B algorithm to generate opinions. In order to do 
so, the true matrices are run through the simSME() function from the GenerateSMEcrit.R 
script within R2B. The simSME() function takes in the true matrix as its first argument 
and N, the number of experts to test, as the second argument. This function generates 
reliable, N-/, expert opinion matrices and one adversarial expert subjective matrix by 
switching the “true” matrix probabilities for excellent with poor and fair with good for 


each alternative. 


3. Function Implementation 


We initialize matrices with N rows and m columns to save the number of times a 
fuzzy evaluation result lands in any of the four linguistic categories. The results appear 
as normalized probabilities. Table 11 illustrates the expert statistics matrix for CLS-5000 
in this study. 100 simulations were conducted for each row of Table 11. The results 
depict the probability of a fuzzy evaluation falling in a certain category starting with two 
expert’s opinions up to 10. Table 11 says that when two experts are present with one that 
is misinformed or adversarial, CLS-5000 has a linguistic grade predominantly in the good 
category. Explicitly, CLS-5000 has 27% membership in the good category. But with 
more informed experts, we find unsurprisingly our original finding with the CLS-5000 


predominantly in the excellent grade. 
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CLS-5000 
EXCELLENT] 6000 | FAIR | POOR 


2 
3 
4 
5 
6 
7 
8 
9 





S 


Table 11. | CLS-5000 saved fuzzy evaluation results by number of experts 


The same steps of multiplying normalized subjective assessment matrices with 
criteria weights in FCE occur using the FuzzyResults() function depicted in Figure 5. The 
results for the 100 iterations of each N ranging from 2 to 10 are saved into a table such as 


Table 11 for each alternative for examination. 


4. Results 


Overall, CLS-5000 is again ranked the “best” alternative for a logistic MOT in 
Table 12. Table 12, depicts the results from the robustness test of FCE results to expert 
bias. This is not a surprising result since our “true” inputs put CLS-5000 as the best. The 
purpose is to check the variability in the results due to the presence of an adversary or 
misinformed individual. Variations mainly occur with lower numbers of experts. The 
results illustrate that an adversary in the presence of only one other opposing expert can 
skew the results in their favor. The combat logistics scenario requires at least two 
experienced experts out of three to assume useful results. Similarly, it is recommended 


that at least two times as many unbiased experts are required for one biased. 


Additionally, the results reveal FCE can be a reasonably robust method even in 
the presence of intentional bias. The ranking for JHSV and LCS-2 are similar to those 
found in Chapter III. Second place is secured by JHSV predominantly in the “good” 
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category. Lastly, LCS-2 teeters between “fair” and “poor.” These values are scaled from 
each alternative’s saved fuzzy evaluation results (as in Table 11) multiplied by row to a 
value scale for each linguistic category. The value scale is from | to 4 with excellent = 4, 


good = 3, fair = 2, and poor = 1. 


WT AVG Scores 


| WTAVGScores 
2|___0.57525] 062375| 0.6115] 
| oes] __0.7325[ 0.5875] 
a|_____owaal__0.81[ 0.47275] 


s| 08s oval _o.a7| 
s|____ 090s] _o.7as[ 0.46] 
7|___osa07s| 0.77 0.4585] 
s|____osi7s|____o.77|__0.as| 
o|____osi7s| ____o7e7s[__0.a7| 
10 





Table 12. Scaled simulated fuzzy evaluation scores from 2 to 10 experts 


This study also found that in general with a greater number of experts providing 
inputs, the closer the results were to the true response. R2B is a way of evaluating the 
potential uncertainty in FCE and showing under what conditions a same alternative is still 
chosen. R2B results are specific and applicable only to the existing scenario. However, 
this method can be applied to any set of expert opinions to estimate the potential effect of 


bias on FCE results. 
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Vv. CONCLUSIONS 


A. SUMMARY 


The FCE method is one of many that solve multiple criteria decision-making 
(MCDM) problems by incorporating quantifiable subject matter expert input. In the 
literature, there are many MCDM techniques; and some are better suited than others 
depending on the decision problem. Additionally, in practice, there is generally no 
standard for how to transform subjective inputs into numbers to prioritize a group of 
alternatives. Practitioners generally meld varying MCDMs with AHP or another 
engineered mathematical method to create a reasonable process for transforming expert 
opinions into quantitative data (Cheng et al., 1999; Dagdeviren et al., 2009; Yan et al., 
2008). This thesis performs its transformations through AHP criteria weights and a fuzzy 
composition method. Cheng (1996) also uses a similar method to evaluate alternative 


missile systems using fuzzy AHP. 


FCE is an appropriate decision aid capable of explicitly examining various 
complex events, decisions, or phenomena. Specifically, this thesis focuses on military 
applications. Potential and current adversaries, along with U.S. allies, all use FCE. 
FCE’s prevalence in the U.S. seems to be growing too (Singh et al., 2013). The analysis 
conducted in this thesis has developed and highlighted potential pitfalls and benefits to 
FCE. The understanding of this knowledge can prove exceptionally useful for friendly or 


adversarial decision support processes. 


For most of the last 40 years, many academics were quick to discredit the 
principles and methodology behind FCE. This is because FCE incorporated imprecise 
and subjective processes. Yet, as researchers continue to find uses for capturing the 
partialities and uncertainties in everyday phenomenon, FCE has become a notable 
methodology. But the subjectivity in inputs and the bias that could be present remains a 


criticism. 


The weaknesses of FCE include the requirement for a well-structured decision 


problem. FCE does not perform well unless correct parameters of a problem are known. 
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Specifically, in order for the AHP in FCE to perform well, it requires a well-mapped 
hierarchy tree with “consistent” subjective inputs to formulate appropriate criteria 
weights. The subjective judgments and rankings from the decision-maker have a large 
influence on the accuracy of FCE results. Accuracy, in statistical terms, is the degree of 
closeness a measurement of a quantity is to its actual “true” value. If subjective 
judgments have large biases, then FCE results are unrealistic and expected to be 
inaccurate. Although AHP relies on a “consistency ratio” to reduce the amount of biased 
error in paired comparisons, the paired comparisons are not resistant to the predisposition 
of results by the decision-maker. Issues arise during group think tanks where expert 
opinions from higher ranked, less knowledgeable officials are accepted over more 
knowledgeable junior officers. Normally, a solution to this problem is the requirement 
for pairwise comparison matrices to be “consistent” with less than 10% of inputs being 
inconsistent. However, subjective judgments can still be skewed when younger experts 
are coerced into reevaluating their original subjective inputs in order to create consistent 
matrices towards decision-makers’ predispositions. While these situations may occur 
rarely, this thesis explores what could happen under different scenarios where expert 


judgments are faulty. 


FCE rankings are not precise without a method for choosing a larger sample size 
of subjective inputs involved in a particular scenario. FCE takes the mode of fuzzy 
evaluations as the solution and disregards the distribution of these evaluations. If the 
alternative has membership scores almost evenly distributed between all the categories 
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from “excellent,” “good,” “fair,” and “poor,” only the category in which the alternative 
has the highest membership prevails. Additionally, FCE results are prone to potentially 
intentional bias. The R2B (robustness to bias) algorithm was created in the R statistical 
language to test the robustness of FCE results to expert bias through simulation. The 
range of experts was chosen based upon the realities of a limited availability of subject 
matter experts in general. R2B provides an estimate of the variability associated with 


FCE outcomes. 


R2B’s algorithm allows users to estimate the number of experts required in order 
to reduce the probability of an incorrect ranking for a particular scenario with one 
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malicious expert. R2B cannot guarantee the accuracy of results. Accuracy in results is 
generally unknown, even with a rigorous study of where the original FCE results came 
from. Nonetheless, R2B can be applied to instances where FCE has already been 
performed. In these situations, R2B shows that the same alternative might still be chosen 
even with added simulated variability and adversarial inputs. Furthermore, the user is 
able to determine the minimum number or ratio of experts required in order to overcome 
any effects from an adversarial expert. However, proper use of R2B requires the user to 
understand handling of alternatives that end up in the same linguistic category. Users are 
required to understand the Max Degree of Membership and Ordered Weighted Averaging 
methods for breaking ties. Unfortunately, users should know that there is no standard for 
dealing with cases in which both methods for breaking ties conflict and lead to separate 


decisions. 


In conclusion, FCE can provide useful information to military leaders and 
decision-makers requesting the optimal selection amongst several alternatives as long as 
its limitations are understood. Moreover, FCE offers solutions taking into consideration 
the imprecision and subjectivity inherent in all real situations. In order to reach this 
endeavor, a well-defined scenario with specified parameters are required for successful 
use of MCDMs such as the AHP. The bias in expert opinion found in the subjective 
inputs in FCE must be carefully considered. This thesis suggests implementing R2B, but 
this is by no means an exhaustive method for dealing with the biasness in expert 
opinions. R2B’s capacity to evaluate variability amid inclusion of a malicious expert has 
implications in a combat logistics scenario. Perhaps FCE combined with other 
mathematical methods such as optimization can be used to support decision support 


processes and aid the decision-maker. 
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